Periodically, management/historical data, including accounting, performance, security and fault logging data (or some portion thereof), may need to be retrieved from distributed modules (i.e., cards, printed circuit boards) within a network device (e.g., switch, router, hybrid switch-router), time stamped and stored in non-volatile memory within the network device. In addition, periodically, this data may need to be taken off the network device and moved to, for example, a workstation for processing and billing integration. The processing applications generally require the data to be in American Standard Code for Information Interchange (ASCII) format and billing/accounting applications generally require the data to be in Automatic Message Accounting/Billing format of the 0122 structure code defined by the Bellcore Automatic Message Accounting Format (AMA/BAF).
“Current” data may be gathered on relatively small time intervals, for example, every 15 minutes, and some finite amount of current data may be stored in non-volatile memory. Moreover, many applications require 24-hour summary data to be gathered at longer intervals, for example, every 6 or 12 hours, and some finite amount of summary data may also be stored in non-volatile memory. The gathering of 24-hour summary data increases data resiliency such that if any current data is lost or cannot be transferred out of the network device before being overwritten, the 24-hour summary data is still available.
In a distributed, large scale network device with potentially many thousands of interfaces simultaneously up and running, an exorbitant amount of bandwidth may be consumed by periodic bursts of management data being passed from remote cards to a central process on one card. Whether the management data shares bandwidth on the control plane or data plane of the network device, there is a potential for the management data transfers to starve out other essential traffic on the control plane or data transfers on the data plane. Also, the act of updating and aggregating management data may take processing cycles away from time-critical processes that reside within the same processor domain.
In addition, the transfer rate of the management data may exceed the rate at which data can be stored in non-volatile memory within the network device. Therefore, a certain amount of buffering of management data at the central process is generally necessary. Buffering, however, consumes memory/storage space within the network device, and process memory within the network device is a finite resource. The larger the amount of management data, the larger the amount of memory space required, and it is possible that once the buffering of management data at a central process reaches the confines of process memory, further incoming management data will be dropped.
Moreover, the rate at which data is stored in non-volatile memory may exceed the rate at which data can be written out of the network device. This may also lead to a need for larger memory and may cause data loss.